Field of the Invention
This invention relates to improvements in respect of a time of flight camera system configured to resolve either or both of the direct path and multipath components of radiation received by the system.
Description of the Related Art
Time of flight camera systems are able to resolve distance or depth information from light radiation beams which have been modulated and reflected from a target in a scene. These camera systems calculate a distance measurement from the target based on phase information extracted from the received reflected radiation.
The measurements generated by the sensors of the time of flight camera represent the direct path reflections of a modulated light source, and a multipath component from the contribution of multiple reflections of the same modulated light source from nearby objects. Time-of-flight imaging suffers from a number of effects that are detrimental to imaging quality. Examples include multi-path interference and motion blur.
The direct path radiation is of interest in depth or range measurement applications while in other cases the multipath component provides valuable information as to the current condition of the target. For example PCT patent specification publication WO2012/053908 discloses a tomographic style application for time of flight technology where the multipath component provides subsurface scattering information.
The contribution of the multipath component is unwanted in time of flight depth sensing applications. Prior art implementations of these devices have been unable to resolve depth information from areas of a scene which include corners or other similar reflecting surfaces. This is due to small difference in (optical path length) distance between the direct and multi-path signal returns received.
In applications involving 3D imaging using structured light scanning multi-path interference is also a source of error to be addressed. In these applications structured light patterns are projected on a target, with deformation of the pattern detected by a receiving camera being used to determine the form or surface shape of the target. This may be contrasted with time-of-flight imaging where the phase of received light is used to determine propagation delay and therefore distance information. Therefore, the measurement error caused by multiple propagation paths in structured light depth cameras and time of flight cameras is different. Time of flight cameras measure the phase and amplitude of the returning light, while structured light methods measure the intensity of the light in the scene.
A number of publications have been made in the 3D imaging field which aim to reduce the error effects of multipath interference. An example of such an approach is published by Shree K Nayar, Gurunandan Krishnan, Michael D Grossberg, and Ramesh Raskar, entitled “Fast separation of direct and global components of a scene using high frequency illumination” ACM Transactions on Graphics (TOG), 25(3):935-944, 2006. In this publication a fast method of separating global and direct light returns is proposed by projecting structured light patterns at a target, with each pattern varying the illumination experienced by each area of the target which is assessed by a pixel of the receiving cameras system. The information gathered by this approach employs spatial variation in the projected patterns to isolate direct path returns for each pixel of the camera.
A further publication related specifically to time-of-flight camera systems and the mitigation of multipath error effects is provided by published US patent application US 2014/0055771 to Oggier. However, although targeted at a time-of-flight camera application, the structured light transmission system disclosed operates to scan only an isolated target of interest within the entire field of view of the camera. The end result of this approach is to multiply the direct path returns received by the camera, making multipath returns a lower relative proportion of the received signal. Although an improvement over the prior art, this approach does not calculate or isolate the direct path component of a received camera signal, simply providing an error reduction effect.
One aspect of these techniques and technologies is the projection of a patterned illumination, as opposed to the traditional homogeneous illumination field. Typically a Digital Micromirror Device (DMD), also known as a Micromirror Array or Digital Light Projector device is used to generate the pattern. DMDs have the disadvantages of large size, large power consumption, and complicated controlling image generation electronics are required. Another significant drawback is inefficiency, as the light not being projected is wasted. However, they have the advantage that one device can generate the multiple patterns needed for many of the techniques and technologies.
It would therefore be of advantage to have improvements over and above prior art time of flight camera systems which address any or all of the above issues. In particular improvements to a time of flight camera system which allowed for the resolution of either or both direct path and multipath radiation contributions in the signals received by the camera system would be of advantage.